Contribuições de aprendizado por reforço em escolha de rota e controle semafórico

A área de sistemas inteligentes de transporte há muito investiga como empregar tecnologias da informação e comunicação a fim de melhorar a eficiência do sistema como um todo. Isso se traduz basicamente em monitorar e gerenciar a oferta (rede viária, semáforos etc.) e a demanda (deslocamentos de pessoas e mercadorias). A esse esforço, mais recentemente, estão sendo adicionadas técnicas de inteligência artificial. Essa tem o potencial de melhorar a utilização da infraestrutura existente, a fim de melhor atender a demanda. Neste trabalho é fornecido um panorama focado especificamente em duas tarefas onde a inteligência artificial tem contribuições relevantes, a saber, controle semafórico e escolha de rotas. Os trabalhos aqui discutidos objetivam otimizar a oferta e/ou distribuir a demanda.The field of of intelligent transportation systems has long investigated how to employ information and communication technologies to improve the efficiency of the system as a whole. This basically means to monitor and manage both supply (traffic network, traffic signals etc.) and demand (vehicles, people and goods). More recently, artificial intelligence techniques are being added to this effort, as they have the potential to improve the usage of existing infrastructure to meet the corresponding demand. In this paper, an overview is given, focusing specifically on two tasks where artificial intelligence has made relevant contributions, namely, traffic signal controls and route choices. The works discussed here aim at optimize the supply and/or distribute the demand

Exploiting Domain Knowledge in Making Delegation Decisions

@inproceedings{conf/admi/EmeleNSP11, added-at = {2011-12-19T00:00:00.000+0100}, author = {Emele, Chukwuemeka David and Norman, Timothy J. and Sensoy, Murat and Parsons, Simon}, biburl = {http://www.bibsonomy.org/bibtex/20a08b683088443f1fd36d6ef28bf6615/dblp}, booktitle = {ADMI}, crossref = {conf/admi/2011}, editor = {Cao, Longbing and Bazzan, Ana L. C. and Symeonidis, Andreas L. and Gorodetsky, Vladimir and Weiss, Gerhard and Yu, Philip S.}, ee = {http://dx.doi.org/10.1007/978-3-642-27609-5_9}, interhash = {1d7e7f8554e8bdb3d43c32e02aeabcec}, intrahash = {0a08b683088443f1fd36d6ef28bf6615}, isbn = {978-3-642-27608-8}, keywords = {dblp}, pages = {117-131}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, timestamp = {2011-12-19T00:00:00.000+0100}, title = {Exploiting Domain Knowledge in Making Delegation Decisions.}, url = {http://dblp.uni-trier.de/db/conf/admi/admi2011.html#EmeleNSP11}, volume = 7103, year = 2011 }Postprin

Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control

In reinforcement learning (RL), dealing with non-stationarity is a challenging issue. However, some domains such as traffic optimization are inherently non-stationary. Causes for and effects of this are manifold. In particular, when dealing with traffic signal controls, addressing non-stationarity is key since traffic conditions change over time and as a function of traffic control decisions taken in other parts of a network. In this paper we analyze the effects that different sources of non-stationarity have in a network of traffic signals, in which each signal is modeled as a learning agent. More precisely, we study both the effects of changing the \textit{context} in which an agent learns (e.g., a change in flow rates experienced by it), as well as the effects of reducing agent observability of the true environment state. Partial observability may cause distinct states (in which distinct actions are optimal) to be seen as the same by the traffic signal agents. This, in turn, may lead to sub-optimal performance. We show that the lack of suitable sensors to provide a representative observation of the real state seems to affect the performance more drastically than the changes to the underlying traffic patterns.Comment: 13 page

Temporal Network Analysis of Literary Texts

We study temporal networks of characters in literature focusing on "Alice's Adventures in Wonderland" (1865) by Lewis Carroll and the anonymous "La Chanson de Roland" (around 1100). The former, one of the most influential pieces of nonsense literature ever written, describes the adventures of Alice in a fantasy world with logic plays interspersed along the narrative. The latter, a song of heroic deeds, depicts the Battle of Roncevaux in 778 A.D. during Charlemagne's campaign on the Iberian Peninsula. We apply methods recently developed by Taylor and coworkers \cite{Taylor+2015} to find time-averaged eigenvector centralities, Freeman indices and vitalities of characters. We show that temporal networks are more appropriate than static ones for studying stories, as they capture features that the time-independent approaches fail to yield.Comment: 17 pages, 11 figure

Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritization

Multi-objective reinforcement learning (MORL) algorithms tackle sequential decision problems where agents may have different preferences over (possibly conflicting) reward functions. Such algorithms often learn a set of policies (each optimized for a particular agent preference) that can later be used to solve problems with novel preferences. We introduce a novel algorithm that uses Generalized Policy Improvement (GPI) to define principled, formally-derived prioritization schemes that improve sample-efficient learning. They implement active-learning strategies by which the agent can (i) identify the most promising preferences/objectives to train on at each moment, to more rapidly solve a given MORL problem; and (ii) identify which previous experiences are most relevant when learning a policy for a particular agent preference, via a novel Dyna-style MORL method. We prove our algorithm is guaranteed to always converge to an optimal solution in a finite number of steps, or an $\epsilon$-optimal solution (for a bounded $\epsilon$) if the agent is limited and can only identify possibly sub-optimal policies. We also prove that our method monotonically improves the quality of its partial solutions while learning. Finally, we introduce a bound that characterizes the maximum utility loss (with respect to the optimal solution) incurred by the partial solutions computed by our method throughout learning. We empirically show that our method outperforms state-of-the-art MORL algorithms in challenging multi-objective tasks, both with discrete and continuous state and action spaces.Comment: Accepted to AAMAS 202

"A Nova Eletricidade: Aplica\c{c}\~oes, Riscos e Tend\^encias da IA Moderna -- "The New Electricity": Applications, Risks, and Trends in Current AI

The thought-provoking analogy between AI and electricity, made by computer scientist and entrepreneur Andrew Ng, summarizes the deep transformation that recent advances in Artificial Intelligence (AI) have triggered in the world. This chapter presents an overview of the ever-evolving landscape of AI, written in Portuguese. With no intent to exhaust the subject, we explore the AI applications that are redefining sectors of the economy, impacting society and humanity. We analyze the risks that may come along with rapid technological progress and future trends in AI, an area that is on the path to becoming a general-purpose technology, just like electricity, which revolutionized society in the 19th and 20th centuries. A provocativa compara\c{c}\~ao entre IA e eletricidade, feita pelo cientista da computa\c{c}\~ao e empreendedor Andrew Ng, resume a profunda transforma\c{c}\~ao que os recentes avan\c{c}os em Intelig\^encia Artificial (IA) t\^em desencadeado no mundo. Este cap\'itulo apresenta uma vis\~ao geral pela paisagem em constante evolu\c{c}\~ao da IA. Sem pretens\~oes de exaurir o assunto, exploramos as aplica\c{c}\~oes que est\~ao redefinindo setores da economia, impactando a sociedade e a humanidade. Analisamos os riscos que acompanham o r\'apido progresso tecnol\'ogico e as tend\^encias futuras da IA, \'area que trilha o caminho para se tornar uma tecnologia de prop\'osito geral, assim como a eletricidade, que revolucionou a sociedade dos s\'eculos XIX e XX.Comment: In Portugues